Automatically Choosing Appropriate Gestures for Jokes

We examine the problem of automatically selecting gestures that are appropriate to use when telling a joke or a short story. Our current application of this is a joke telling humanoid robot that needs to be able to select natural gestures for arbitrary input. The topic is important because humans use body language and gestures, thus socially interactive robots should also be able to do so for more natural interaction. We asked evaluators to assign appropriate gestures from a set of gestures the robot can perform to 50 jokes from a corpus of jokes in Japanese. We then evaluated different methods for automatically selecting gestures on this data set. While human inter-agreement was rather low, indicating that this is a fairly difficult task and that some jokes have no obviously fitting gesture, the best method performs on par with humans and clearly outperforms the baseline.